Investigation of 360-degree instrumentation effects: application of the Rasch rating scale model.
نویسندگان
چکیده
Performance appraisals have been frequently investigated for inaccuracies attributable to raters. In addition to rater contributions, problems in measurement can arise from properties of instrumentation. If administered items do not parallel the full range of employee performance, a restriction of range can occur. Here employees are rated similarly (no distinction made) not because of rater error, but because of an instrument floor or ceiling effect. A RASCH measurement procedure was applied to a 360-degree dataset in order to uncover potential instrumentation effects. A similar magnitude ceiling effect was found across rater categories. It is recommended that performance appraisal practitioners consider including items of greater difficulty in their criterion measures.
منابع مشابه
Life-wise Language Learning Textbooks: Construction and Validation of an Emotional Abilities Scale through Rasch Modeling
Underlying the recently developed notions of applied ELT and life syllabus is the idea that language classes should give precedence to learners’ life qualities, for instance emotional intelligence (EI), over and above their language skills. By so doing, ELT is ascribed an autonomous status and ELT classes can lavish their full potentials to the learners. With that in mind, this study aimed to d...
متن کاملThe effects of the violation of local independence assumption on the person measures under the Rasch model
Local independence of test items is an assumption in all Item Response Theory (IRT) models. That is, the items in a test should not be related to each other. Sharing a common passage, which is prevalent in reading comprehension tests, cloze tests and C-Tests, can be a potential source of local item dependence (LID). It is argued in the literature that LID results in biased parameter estimation ...
متن کاملRater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model
In this study, the researcher used the many-facet Rasch measurement model (MFRM) to detect two pervasive rater errors among peer-assessors rating EFL essays. The researcher also compared the ratings of peer-assessors to those of teacher assessors to gain a clearer understanding of the ratings of peer-assessors. To that end, the researcher used a fully crossed design in which all peer-assessors ...
متن کاملA Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool
The studies conducted so far on the effectiveness of resolution methods including the discussion method in resolving discrepancies in rating have yielded mixed results. What is left unnoticed in the literature is the potential of discussion to be used as a training tool rather than a resolution method. The present study addresses this research gap by exploring the data coming from rating behavi...
متن کاملThe Impact of Raters’ and Test Takers’ Gender on Oral Proficiency Assessment: A Case of Multifaceted Rasch Analysis
The application of Multifaceted Rasch Measurement (MFRM) in rating test takers’ oral language proficiency has been investigated in some previous studies (e.g., Winke, Gass, & Myford, 2012). However, little research so far has ever documented the effect of test takers’ genders on their oral performances and few studies have investigated the relationship between the impact of raters’ gender on th...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Journal of applied measurement
دوره 8 4 شماره
صفحات -
تاریخ انتشار 2007